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FOREWORD 


A  major  mission  of  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  is  to  find  ways  to  improve  Army 
training.  The  application  of  technological  developments  to 
training  is  one  of  those.  Advances  in  computer  hardware  and 
software  at  diminishing  cost  have  opened  up  new  opportunities  for 
computer-based  training.  The  work  reported  in  this  report 
explores  the  application  of  ideas  in  this  area.  The  findings 
will  be  of  interest  not  only  in  the  training  community  of  the 
Army  but  also  in  business  and  industry. 


EDGAR  M.  JOHNSON 
Director 
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INTERACTIVE  HYPERMEDIA  FOR  TACTICAL  TRAINING 


EXECUTIVE  SUMMARY 


Requirement: 

Field  training  exercises  at  the  National  Training  Center 
generate  a  wealth  of  data.  A  significant  need  exists  to 
determine  how  these  data  can  be  used  to  improve  training  for  Army 
personnel . 


Procedure : 

This  effort  applied  the  technology  of  hypermedia  to  the 
problem  using  readily  available  and  affordable  computer  software 
technologies.  The  data  produced  during  a  large-scale  training 
exercise  was  structured  into  a  hypermedia-based  proof-of- 
principle  system  for  training  ground  war  tactics. 


Findings: 

The  potential  of  the  technology  was  successfully 
demonstrated.  The  prototype  lesson,  which  runs  on  a  286-based 
MS-DOS  computer,  features  hypermedia  structuring  of  textual 
information  and  high-resolution  color  static  and  dynamic 
graphics.  The  lesson  includes  training  objectives  and  an 
independent  evaluation  and  feedback  component. 


Utilization  of  Findings: 

The  method  employed  shows  the  high  potential  of  the 
hypermedia-based  approach  to  the  synthesis  of  training  exercise 
data  for  training.  The  findings  of  this  effort  will  contribute 
in  several  ways  to  future  work  in  this  area. 

1.  Future  work  can  build  directly  upon  the  progress 
achieved  in  this  project. 

2.  Guidelines  are  presented  for  resources  necessary  for  a 
full-scale  development  of  a  tactical  training  system 
based  on  interactive  hypermedia  technology. 

3.  This  project  shows  the  value  of  a  hypertextual  approach 
as  a  way  of  organizing  and  integrating  diverse  types  of 
training  exercise  data  for  use  in  computer-based 
training . 
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INTERACTIVE  HYPERMEDIA  FOR  TACTICAL  TRAINING 

INTRODUCTION 

Consider  the  following  training  situation: 

A  lieutenant  sits  at  a  personal  computer. 

He  depresses  the  computer's  mouse  button,  click... 

-  reads  the  Brigade  Mission  Order  of  an  actual  training 

exercise  conducted  at  the  National  Training 
Center,  click... 

-  reads  the  Battalion  Task  Force  Mission  Order,  click. . . 

-  views  a  terrain  map  with  maneuver  graphics,  click... 

-  reads  the  Operation  Order  of  the  Task  Force,  click. . . 

-  views  a  time-tagged  replay  of  the  battle  with  color- 

coded  vehicles  moving  across  a  display  of  3-D 
terrain,  click. . . 

-  views  a  video  clip  of  the  first  contact  with  the 

Opposing  Force,  click. . . 

-  hears  a  sound  bite  of  the  exercise  Observer-Controller 

critiquing  the  battle  and  discussing  with  Opposing  Force 
Conunander  what  his  objectives  were,  click.  .  . 

-  sees  a  chart  cf  Battle  Damage  statistics,  click... 

-  is  asked  a  question  about  the  adequacy  of  the  Fire 

Support  planning  of  the  unit  being  trained, 
click. . . 

-  reviews  the  Operation  Order,  click. . . 

-  responds  to  a  question  and  receives  feedback  about 

his  answer  and  about  his  performance  on  the 
training  module  he  has  just  completed. 

The  lieutenant  smiles,  he  did  well  and  he  likes  it! 

This  vignette  illustrates  the  functionality  of  an  effective, 
interactive,  computer-based  training  system  for  military  tactics. 
The  technology  for  realizing  the  setting  describes  is  both  at 
hand  today  and  affordable.  Indeed,  most  of  the  features 
illustrated  exist  in  the  prototype  training  system  that  is  the 
subject  of  this  paper. 

Army  Training 

The  U.S.  Army  expends  tremendous  effort  and  resources  to 
assure  that  its  personnel  are  adequately  trained.  Training 
and  skill  sustainment  are  a  complex  and  continuous  endeavor 
because  of  many  factors,  including  the  implementation  of  new 
systems,  personnel  turbulence  and  skill  decay,  and  an 
enormous  variety  of  specific  jobs  to  be  trained.  Not  only 
must  soldiers  and  leaders  be  proficient  in  their  individual 
specialties,  but  it  is  essential  that  they  be  capable  of 
working  together  as  crews,  teams  and  progressively  larger 
units.  The  Army  refers  to  the  training  of  groups  as 
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collective  training.  Today  all  of  these  challenges  must  be 
met  under  the  shadow  of  diminishing  resources. 

As  a  part  of  collective  training,  Army  units  train  to  operate 
in  a  combined  arms  and  services  environment.  Army  Combat,  Combat 
Support  and  Combat  Service  Support  branches  must  all  function 
together  with  the  personnel,  high  technology  weaponry  and  systems 
from  other  services  in  a  highly  integrated  manner  to  survive  on 
the  battlefield  and  accomplish  the  mission.  To  prepare  for  such 
integrated  operations  requires  extensive  training  in  a  simulated 
combat  environment. 

The  Army  conducts  collective  training  operations  at  its  Combat 
Training  Centers.  The  National  Training  Center  (NTC)  is  prime 
facility  for  large-scale  armor  and  mechanized  infantry  combat 
training.  It  is  the  most  developed  of  the  Combat  Training 
Centers.  It  is  located  in  the  Mojave  Desert  at  Fort  Irwin  near 
Barstow,  California.  The  NTC  consists  of  over  1,000  square  miles 
of  terrain  accommodating  large  maneuver  and  live  fire  exercises 
plus  nap-of-the-earth  flying,  firing  of  air  defense  weapons  and 
practice  in  the  use  of  electronic  warfare. 

Since  1981,  training  at  the  NTC  has  focused  on  the  battalion  task 
force  level  and  has  recently  expanded  to  include  support  for 
brigade  operations.  A  permanent  Opposing  Force,  a  full-time 
cadre  of  observer-controllers,  consisting  of  some  of  the  best 
commissioned  and  noncommissioned  officers  in  the  Army,  and  an 
instrumented  simulated  battlefield  (Sulzen,  1986)  contribute  to 
the  realism  and  value  of  the  training  the  visiting  unit 
experiences  at  the  NTC  during  a  typical  two  weeks  of  on-site 
training.  Extensive  training  feedback  and  analysis  are  provided 
to  maximize  the  improvement  in  tactical  capability.  The 
performance  of  units  at  the  NTC  is  also  used  to  provide  senior 
commanders  a  measure  of  the  Army's  readiness  for  combat  (U.S. 
General  Accounting  Office,  1991) . 

Training  operations  at  the  NTC  have  been  important  in  identifying 
common  training  deficiencies  among  Army  units  (U.S.  General 
Accounting  Office,  1984)  and  in  providing  a  data  source  upon 
which  to  base  improvements  in  training,  organization,  equipment 
and  operational  doctrine.  The  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  has  established  an  archive  of 
data  generated  by  these  training  exercises  conducted  at  the 
Combat  Training  Centers.  It  develops  methodologies  for 
utilization  and  supports  a  wide  range  of  analyses  conducted  with 
sponsorship  of  interested  agencies  throughout  not  only  the  Army 
but  other  military  services  and  non-defense  agencies. 

One  of  the  challenges  of  Army  training  is  how  to  exploit 
modern  technological  advances  to  improve  the  effectiveness 
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and  efficiency  of  training,  especially  collective  training. 
There  have  been  significant  advances  in  both  learning 
technologies  and  computer  sciences  which  together  offer 
important  contributions  not  only  to  Army  training  but  to 
training  in  other  institutional  settings.  One  such  advance 
serves  as  the  foundation  for  the  project  reported  here. 

Terminology 

The  origins  of  the  concept  of  connecting  ideas  elec¬ 
tronically  in  an  other  than  sequential  way  are  traceable 
back  nearly  fifty  years.  An  automated  information  access 
scheme  was  envisioned  which  included  the  capability  of  fast 
accessing  and  linking  of  information,  of  storing  the  trail 
of  links,  and  of  annotating  the  retrieved  information  (Bush, 
1945).  Englebart  (1963)  discussed  the  functionalities 
necessary  for  computers  to  augment  human  abil ities  including 
links  between  texts,  document  libraries,  and  separate 
private  space  for  computer  users'  personal  files,  computer 
screens  with  multiple  display  windows,  and  the  facilitation 
of  work  done  by  multiple  persons  working  in  collaboration. 
Later,  he  introduced  the  mouse  computer  pointing  device, 
outliner  and  idea  processor,  and  on-line  help  systems. 

The  term  hypertext  was  coined  by  T.H.  Nelson  in  1965.  The 
simplest  definition  is  that  hypertext  is  “nonsequential 
writing  or  reading"  (Nelson,  1987) .  Other  writers  (Slatin, 
1991)  have  emphasized  that  a  true  hypertext  can  only  be 
realized  in  a  computer-based  system,  that  it  is  a  system  the 
existence  of  which  is  completely  dependent  upon  a  computer. 
When  the  concept  of  hypertext  was  first  implemented  in 
operational  programs  they  contained  textual  material 
exclusively  simply  because  textual  display  were  the  first 
non-numerical  output  of  early  computers. 

In  a  broader  context,  hypertext  can  be  thought  of  as  a  mode 
of  information  organization  which  is  like  an  n-dimensional 
web.  Items  of  information  are  associated  without  bound. 

What  is  important  about  it  is  that  a  hypertext  user  or 
reader  can  proceed  through  the  information  in  a  highly 
individual  way  consistent  with  his  or  her  information 
requirements.  In  fact,  a  personal  system  of  information 
storage  based  on  just  this  principle  of  information 
organization  has  been  recently  implemented  (Phillips,  1991) . 

As  a  generalization  of  hypertext,  hypermedia  is  a  technology 
for  organizing  discrete  quantities  of  information,  termed 
information  nodes,  for  utilization  in  an  arbitrary, 
nonsequential  manner  by  traversing  links  between  information 
nodes.  Conceptually,  an  information  node  is  unrestricted  as 
to  type  of  information  or  amount  of  information  it  may 
contain.  It  may  consist  of  textual  material,  still  or 
moving  pictures,  video  and/or  sound  recordings,  independent 
computer  programs,  or  other  entire  hyperdocuments.  A  link 
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is  an  association  between  nodes  enabling  the  user  to  move 
from  the  current  node  to  another.  A  hyperdocument  is  a 
structure  of  nodes  and  links  which  is  self-contained, 
dealing  with  a  particular  subject  domain  for  some  specific 
purpose.  These  concepts  are  well  developed  and  extensive 
discussions  are  available  (Nielsen,  1990,  Berk  &  Devlin, 
1991)  . 

Simple  interaction  is  implicit  in  the  concepts  of  hypertext 
and  hypermedia.  Therefore,  referring  to  interactive 
hypermedia  might  appear  redundant  except  for  the  explicit 
emphasis  placed  on  the  concept  of  interaction. 

Interactivity  in  this  context  has  been  defined  as  "the 
incorporation  of  repetitive,  frequent  and  meaningful 
iterations  of  a  sti-..ulus-response-analysis-feedback  cycle 
into  material  that  is  presented  in  a  medium  that  permits  it" 
(Gery,  1987) .  Further,  the  sort  of  interaction  necessary — 
making  choices  regarding  navigation — distinguishes  use  of  a 
hypertextually  organized  information  base  from  the  highly 
structured  stimulus-response-analysis-feedback  cycle  which 
characterizes  interaction  in  a  typical  computer-based 
training  system. 

Several  dimensions  of  interactivity  computer-based  systems 
have  been  identified:  (1)  the  nature  and  complexity  of  the 
interactions  (questions  or  decisions,  fixed  or  open-ended 
options,  inherent  complexity  of  required  responses) ,  (2) 

response  analysis  and  feedback  (nature,  depth,  programming 
of  anticipated  or  unanticipated  responses,  tolerance  for 
variable  input) ,  (3)  branching  (amount,  conditionality) ,  and 

(4)  degree  of  learner  versus  trainer  control.  Gery  (1987) 
discusses  detailed  considerations  for  measuring  the  degree 
of  interactivity  between  learner  and  system. 

The  concept  of  interactive  hypermedia  emphasizes  the  fact 
that  the  user  actively  participates  in  utilization  of  a 
system  that  possesses  a  hypertextual  organization  of 
multiple  media.  The  payoff  of  such  a  system  is 
parsimoniously  expressed  by  the  ancient  Chinese: 

I  hear,  I  forget. 

I  see,  I  remember. 

I  do,  I  understand. 

Findings  of  modern  psychological  research  substantiate  this 
proverb  (Laurillard,  1987;  Packer,  1988;  Sculley,  1990). 
Learners  progress  faster,  retain  more,  and  enjoy  it  more 
when  they  are  meaningfully  involved  in  the  process. 

What  is  novel  about  the  current  time  is  that  the 
technologies  have  now  matured  sufficiently  that  these 
techniques  are  readily  available  for  application  in  training 
systems  and  direct  application  to  the  training  environment 
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can  be  made  using  currently  available  hardware  and  software 
at  a  realistic  cost  (Hannafin  and  Peck,  1988) . 


Technological  Leverage 

Constrained  resources  is  emerging  as  the  keynote  of  the 
decade.  In  the  arena  of  Army  training  as  elsewhere  the 
message  is  to  make  do  with  less.  One  of  the  ways  of  moving 
toward  this  goal  is  to  take  advantage  of  technological 
advances  when  they  produce  higher  quality  output  at  the  same 
or  reduced  cost,  or  same  output  at  reduced  cost.  Using 
technology  for  this  purpose  has  been  termed  technological 
leverage. 

The  application  of  computers  for  training  has  held  such 
promise  for  decades.  The  effectiveness  of  interactive 
videodisc  technology  in  certain  Department  of  Defense 
training  systems  has  been  established  in  an  extensive  meta¬ 
analysis  (Fletcher,  1991;  Fletcher,  1990) .  Therefore,  the 
effectiveness  of  interactive  hypermedia,  of  which 
interactive  videodisc  is  merely  one  form,  can  reasonably  be 
assumed.  Thus,  the  time  for  fulfillment  of  the  promise  of 
using  computers  for  more  effective  training  is  now  at  hand. 
Furthermore,  because  delivery  costs  continue  to  decrease, 
quality  training  using  computers  and  related  technologies  is 
becoming  more  and  more  affordable.  In  the  area  of 
information  technology,  the  power  is  doubling  at  half  the 
cost  every  two  years  on  the  average  (Feigenbaum,  McCorduck  & 
Nii,  1988). 

Project  Goals 

The  purpose  of  the  research  reported  here  was  to  explore  the 
integration  of  existing  training  exercise  data  from  the  NTC 
into  a  functioning  computer-based  system  employing 
interactive  hypermedia  technology.  It  is  also  intended  to 
provide  a  preliminary  evaluation  of  the  principle  of 
coupling  this  technology  and  the  NTC  data  to  provide 
effective  training  in  fundamental  ground-warfare  military 
tactics.  The  desired  outcome  was  to  develop  a  proof-of- 
principle  system  (POPS)  for  evaluation  of  this  training 
methodology. 


METHOD 

One  of  the  greatest  strengths  of  interactive  hypermedia  for 
training  is  that  the  learner  is  an  active  participant  in  the 
training  process.  Beyond  making  responses  and  receiving 
feedback  the  learner  actively  structures  the  secjuence  in 
which  the  material  is  encountered.  The  metaphor  guiding  the 
project  from  its  beginning  is  the  idea  of  placing  the 
learner  into  the  highly  stimulating  environment  of  an 
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intellectual  playpen.  The  envisioned  environment  has 
boundaries  but  the  concept  is  to  have  it  filled  with  all 
sorts  of  wonderful  and  stimulating  information  objects  so 
that  the  learner's  biggest  problem  at  any  point  is  what  to 
explore,  experience  and  learn  about  next.  The  assessment 
and  feedback  module  assures  that  all  material  will  be 
mastered  to  standard  prior  to  completion  by  the  learner, 
while  leaving  the  ordering  largely  to  the  choices  of  the 
learner. 

POPS  was  designed  as  a  single  module  requiring  the  learner 
approximately  two  hours  to  complete.  The  target  learner  was 
envisioned  as  a  junior  Army  leader  familiar  with  basic 
military  terminology.  Emphasis  was  placed  on  exploring  the 
principles  and  the  range  of  possibilities  of  the 
technologies  involved  rather  than  on  developing  an 
operational  training  system. 

The  targeted  delivery  platform  is  the  286-based  MS-DOS 
computer  with  EGA  graphics  and  Winchester  drive.  These  are 
widely  available  throughout  the  target  population  and  seem 
to  offer  a  reasonable  compromise  between  functionality  and 
cost  at  this  time.  However,  because  technological  advances 
occur  quite  regularly,  determination  of  an  appropriate 
targeted  delivery  platform  will  soon  need  to  be  readdressed. 

Training  Exercise  Selection 

To  focus  the  development  effort  on  a  single  training  event, 
the  assistance  of  a  senior  observer/controller  at  the  NTC 
was  obtained.  Several  exemplar  training  events  were 
identified  and  finally  one  selected  for  use  in  POPS. 

One  characteristic  distinguishing  the  selected  training 
event  is  that  the  same  mission  was  executed  twice  on 
successive  training  days  by  the  training  unit.  The  first 
day,  the  unit  failed  to  accomplish  its  mission.  On  the 
second  day,  it  performed  the  mission  almost  flawlessly  and 
did  succeed.  Thus,  from  a  training  perspective  the  selected 
exercise  provides  an  illustration  of  the  same  mission  being 
carried  out  ineffectively  as  well  as  effectively,  inviting 
direct  comparisons  between  the  two. 

Another  feature  associated  with  the  chosen  training  exercise 
is  that  it  has  a  relatively  complete  data  set  compared  to 
other  training  exercises  under  initial  consideration.  Data 
completeness  enables  a  thorough  understanding  of  what 
occurred  during  the  training  exercise  and  offers  a  basis  for 
clear  judgments  of  the  training  significance  of  events  that 
occurred . 

The  senior  observer/controller  who  aided  in  selecting  the 
training  exercise  used  for  the  development  also  assisted  in 
identifying  the  task-force-level  tactical  training  lessons 


learned.  These  lessons  learned  became  the  key  training 
objectives  around  which  the  lesson  submodules  were  produced. 

Information  Sources 

The  NTC  training  exercise  used  in  the  development  of  POPS 
provided  several  different  types  of  information  which, 
together  constitute  the  training  exercise  data  set.  Each 
type  of  data  will  now  be  described.  For  a  more  thorough 
discussion  of  data  generated  at  the  Combat  Training  Centers, 
including  at  the  NTC  see  Hamza  (in  press). 

Instrumented  data  at  NTC  are  collected  through  a  telemetry 
system.  They  include  digital  records  of  vehicle  positions, 
firing  and  engagement  information  of  instrumented  players, 
and  several  other  types  of  information  pertinent  to  the 
specific  training  event.  The  development  of  POPS  used  these 
data  collected  at  one-  and  five-minute  intervals.  Data  for 
air  players  based  on  10-second  intervals  were  also  explored. 

These  data  are  very  useful  for  studying  the  details  of 
player  movements  and  unit  formations.  Several  software 
systems  have  been  developed  to  replay  the  NTC  battles  using 
these  data.  However,  transient  behavior  of  the  telemetry 
system  coupled  with  terrain  phenomena  introduce  aperiodic 
anomalies.  Also,  not  all  vehicles/players  participating  in 
the  training  exercise  are  instrumented. 

Take-Home  Packages  (THPs)  are  textual  descriptions  and 
evaluations  of  the  training  unit's  performance  during  the 
exercise  for  various  echelons  and  subunits  that  comprise  the 
battle  task  force.  Tables  provide  quantified  battle  damage 
assessmo-nt  data.  The  THP  data  are  particularly  useful  for 
providing  a  comprehensive  perspective  of  the  training 
exercise  and  the  specific  lessons  learned  for  each  of  the 
elements.  These  documents  are  intended  primarily  to  provide 
the  unit  a  training  focus  upon  its  return  to  home  station. 
Much  of  the  THP  information  is  general  in  its  observations 
and  recommendations  concerning  what  occurred  during  the 
training  exercise. 

Video  tape  recordings  were  available  of  After  Action  Reviews 
(AARs)  following  completion  of  the  training  exercise.  These 
are  conducted  at  various  echelons  within  hours  of  the 
exercise  conclusion  and  recorded  without  editing.  An  after 
action  review  is  essentially  a  debrief  of  the  exercise 
participants  where  intentions,  actions,  and  outcomes  are 
reviewed.  Causes  and  alternatives  are  also  discussed.  AARs 
are  lead  by  the  observer/controller  responsible  for  that 
particular  element.  AARs  are  the  best  source  for  obtaining 
information  about  the  reasons  behind  a  particular  aspect  of 
the  training  event  but  are  sometimes  of  a  quality  that 
precludes  extracting  detailed  graphical  information. 
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Audio  con.muni  cat  ions  are  recorded  on  40  channels  with  time 
tagging.  This  is  a  rich  source  of  detailed  information  on 
various  radio  nets  used  during  the  training  event  but  is 
resource  intensive  to  utilize  fully. 

Additional  information  about  the  training  exercise  was 
collected  through  interviews  with  personnel  who  participated 
in  the  training  exercise  and  with  personnel  who  are 
generally  knowledgeable  about  NTC  training  operations.  This 
information  was  valuable  in  obtaining  an  overview 
understanding  of  the  training  event  and  a  framework  for 
integrating  the  details  of  the  simulated  battle. 

Finally,  several  additional  data  sources  were  available  and 
used  in  developing  POPS.  These  included  acetate  overlays  to 
maps  and  a  variety  of  paper-based  information  such  as  orders 
and  task  organization  listings. 

The  data  set  contains  an  extensive  record  of  the  events  that 
occurred  during  the  training  exercise.  It  is  sometimes 
incomplete  and  occasionally  ambiguous  or  erroneous  in  minor 
ways.  Frequently  it  is  necessary  to  infer  the  cause,  reason 
or  purpose  behind  a  particular  action  or  event.  The 
possibility  of  making  an  erroneous  inference  are  small,  but 
not  non-zero.  The  reasons  behind  particular  events  in  the 
training  exercise  record  are  of  substantial  importance  from 
a  training  perspective. 

Design  Considerations 

Although  the  training  exercise  data  set  contains  extensive 
audio  and  video  material,  the  decision  was  made  at  the 
design  phase  of  the  project  to  exclude  audio  and  video 
capabilities  from  POPS  for  reasons  of  economy.  However,  the 
information  contained  in  these  data  sources  can  still  be 
used  although  in  a  textual  medium.  Vocal  audio  recordings 
can  be  transcribed  to  text  and  video  material  can  be 
represented  in  various  graphical  forms  or  described 
textually . 

No  hardware  acquisition  was  necessary  in  support  of  the 
project.  MS-DOS  computers,  scanners  and  a  variety  of 
software  were  available  and  used  in  the  project. 

The  assumption  was  made  at  the  initiation  of  the  project 
that  the  training  exercise  data  set  is  sufficiently  complete 
and  rich  enough  to  develop  POPS.  Based  on  the  data  it  is 
necessary,  first,  to  determine  what  occurred  during  the 
training  exercise,  next,  to  provide  a  basis  for  inferring 
the  reasons  behind  occurrences.  Finally,  it  is  necessary  to 
establish  the  training  relevance  and  significance  of  what 
happened. 
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Software 


In  order  to  implement  POPS,  selection  of  suitable  computer- 
based  training  software  was  necessary.  More  than  70  MS-DOS 
based  authoring  systems  are  on  the  market  and  nearly  as  many 
development  tools  exist  for  the  Microsoft  Windows 
environment  (Wilkens,  1991) .  While  capabilities  of  such 
systems  change  at  a  rapid  pace,  none  meets  every  purpose. 

Any  decision  involves  tradeoffs  among  alternatives.  Gery 
(1987)  identifies  three  dimensions  of  tradeoffs — 
productivity  versus  creativity,  structure  versus  freedom, 
and  power  versus  simplicity — and  discusses  these 
considerations  in  detail.  She  identifies  a  number  of 
additional  factors  along  which  software  systems  vary: 
training  needs  and  the  training  audience;  system  features, 
functions  and  requirements;  authoring  features,  support  and 
training;  costs;  vendor  capabilities;  existing  courseware; 
and  consultation  services. 

The  authoring  system  software  chosen  for  this  implementation 
was  HyperWriter  (Ntergaid,  1990) .  It  satisfied  the 
paramount  criteria  for  the  project:  authoring  and  system 
capabilities  and  requirements,  and  immediate  availability. 

In  addition,  three  specialized  stand-alone  software  systems 
were  utilized  in  the  development  of  POPS.  The  first  was  a 
graphic  tool  which  supports  the  merging  of  data  from  two 
graphic  files  into  a  single  file  for  display  on  the  screen.* 
It  is  used  to  automate  the  insertion  of  terrain  labels  and 
time  stamps  into  graphics  of  the  type  shown  in  Figures  8 
through  11. 

Second,  a  routine  was  adapted  from  another  software  system, 
the  TBAT/MPART  (Nichols  &  Shillcock,  1990)  into  a  stand¬ 
alone  mode  to  present  an  animated  display  of  the  trace  of 
the  battle  on  a  three-dimensional  terrain  representation.^ 

Third,  the  assessment  and  feedback  module  required  a  level 
of  interaction  and  learner-generated  information  storage  not 
supported  by  HyperWriter.  Therefore,  a  text-based  module 
was  developed  to  provide  the  requisite  functionalities  which 
include:  (1)  multiple  entry  points  depending  upon  from 

where  it  is  called  in  the  lesson,  (2)  maintenance  of  learner 
records  using  a  learner-generated  password,  (3)  presentation 
of  fixed  response  items  and  context-dependent  responses,  (4) 
capability  of  being  exited  for  return  to  lesson  at  any  point 
with  restart  at  the  same  position,  and  (5)  presentation  of 
learner  performance  record  and  current  progress  through 


'  This  routine  was  developed  by  Richard  Crenshaw,  ARI  His  assiitance  is  appreciated. 
^  This  software  was  adapted  by  Jack  Briscoe.  BDM  Inc.  His  assistance  is  appreciated. 
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assessment  and  feedback  module  including  identification  of 
sections  of  lesson  requiring  completion. 


RESULTS 

The  functioning  POPS  is  comprised  of  129  nodes  with  nearly 
300  links  among  them.  POPS  incorporates  two  stand-alone 
software  entities  into  the  commercial  authoring  system. 
Approximately  two  and  a  half  megabytes  of  disk  storage  are 
used  for  the  system  and  its  various  supporting  data  files. 
Each  of  the  distinct  types  of  display  is  now  described  and 
illustrated  to  provide  a  detailed  understanding  of  POPS. 

Textual  Material 


Several  types  of  textual  material  can  be  distinguished 
within  POPS.  First  is  textual  data  directly  from  the  data 
source  with  minor  editing,  such  as  correcting  a  misspelling 
or  expunging  a  date  or  other  identification.  Figure  1  shows 
an  example  of  the  display  of  such  textual  material  as  it 
appears  on  the  screen.  This  particular  example  was  taken 
from  the  written  take-home  package  for  the  training 
exercise.  A  transcription  from  an  audio  or  video  sound 
track  would  have  an  identical  foirmat. 

Observe  the  small  box  in  the  upper  right  corner  of  Figure  1. 
It  is  termed  a  note  in  the  nomenclature  of  the  authoring 
system.  The  link  to  the  note  was  traversed  by  placing  the 
arrow  cursor  on  the  abbreviation  "R&S”  and  activated  by 
depressing  the  mouse  button  or  the  appropriate  key  on  the 
keyboard.  In  Figure  1  other  links  are  indicated  where 
triangular  symbols  enclose  text.  The  information  in  the 
note  shown  in  Figure  1  functions  like  a  footnote,  giving 
additional  information.  It  illustrates,  generally,  the 
second  kind  of  textual  information,  which  provides 
explanatory  or  interpretative  information  which  is  directly 
based  on  the  data  or  an  interpretationof  the  data  but  which 
is  not  actually  a  part  of  it. 

There  is  another  possible  type  of  textual  information  which 
has  not  been  employed  extensively  in  POPS.  This  is 
simulated  data.  Where  a  critical  item  of  information  is 
unavailable  in  the  training  exercise  data  sources,  for 
example,  a  portion  of  the  detail  of  the  planning  process,  it 
is  often  possible  to  fabricate  the  missing  information  in 
reasonably  accurate  fashion  based  upon  generalized 
experience  with  NTC  operations  and  the  specific  training 
environment  and  situation  of  the  training  unit.  The 
rationale  for  such  simulated  data  is  to  provide  the  learner 
a  complete  and  consistent  picture  of  the  training  event 
without  the  confusion  of  these  data  problems. 
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Figure  1.  Screen  display  of  textual  information  with  activated  note  link. 


static  Graphics 


Five  types  of  static  graphics  can  be  distinguished  in  POPS. 
They  are  terrain  maps,  data  graphs,  system  linkage  maps, 
adorning  graphics  and  responsive  graphics. 

Maps  are  an  integral  part  of  any  military  planning  and  of 
the  understanding  of  a  military  operation.  Several 
varieties  are  included  in  POPS.  Figure  2  shows  a  large- 
scale  display  of  the  NTC  used  to  show  the  general 
operational  sectors,  axes  of  advances  and  phase  lines  for 
the  training  event.  Color  coding  provides  a  representation 
of  the  gross  geographical  features  of  the  NTC  and  five 
kilometer  grid  lines  and  map  reference  coordinates  are 
presented.  Color  has  been  shown  to  contribute  substantially 
to  data  presentation  (Hudson,  1985) .  The  graphics  on  this 
map  were  constructed  from  acetate  overlays  included  in  the 
training  event  data  set. 

Figure  3  shows  a  topographical  map  which  was  scanned  into  a 
monochrome  digital  format  from  a  standard  military  terrain 
map  and  subsequently  annotated  to  make  key  terrain  features 
more  apparent.  Contour  intervals  are  20  meters.  Major 
anthropogenic  features  such  as  roads  are  shown  as  well  as 
natural  features  such  as  streams. 

A  second  type  of  static  graphic  included  in  POPS  is  data 
graphs  generated  by  other  software.  The  input  is  various 
statistical  data  from  the  training  event  data  set  and  the 
output  is  a  screen  display.  Figure  4  shows  a  screen  of 
information  precomputed  by  a  popular  spreadsheet  program  and 
saved  as  a  graphic  display.  Such  data  displays  are  shown 
on  the  screen  in  color  which  provides  an  additional  channel 
for  encoding  training  relevant  information  such  as 
emphasizing  some  aspect  of  the  information. 

A  third  type  of  static  graphic  display  available  in  POPS  is 
a  system  linkage  map,  in  some  hypertext  systems  termed  a 
graphical  browser.  Figure  5  shows  a  sample  screen  of  the 
system  link  map.  This  display  shows  the  structure  of  links 
between  all  nodes  in  the  currently  open  file.  It  scrolls  to 
display  various  parts  of  the  hyperdocument  and  supports 
interaction  with  the  user.  When  a  node  is  selected,  the 
user  sees  the  window  title,  type  of  link,  and  if  the  node  is 
textual,  the  first  few  lines  of  text,  as  shown  in  the 
figure.  The  user  can  actuate  any  link  displayed  on  the  map 
and  consequently  move  directly  to  that  node.  Thus,  the  link 
map  provides  a  random  access  means  for  navigation  of  the 
hyperdocument.  Although  the  form  of  the  link  map  shown  in 
the  figure  is  specific  to  the  hypermedia  software  used,  the 
principle  is  general  and  most  hypermedia  packages  support  a 
similar  functionality. 
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Figure  5.  Screen  display  of  graphic  showing  system  link  map. 


The  fourth  type  of  static  graphics  is  adorning  graphics. 
These  are  intended  to  give  the  screen  visual  appeal  and  to 
maintain  the  interest  and  motivation  of  the  learner. 

The  fifth  type  of  static  graphics  used  in  POPS  is  responsive 
graphics.  Responsive  graphics  are  characterized  by 
producing  an  action  in  the  system  when  activated.  Typically 
they  are  an  area  of  the  screen  where,  when  the  cursor  is 
placed  within  the  boundary  and  the  mouse  button  or  key  is 
depressed,  some  activation  of  the  software  occurs.  The 
sensitive  area  of  the  screen  is  typically  graphically 
visible  to  the  learner,  as  in  form  of  an  icon  such  as  a 
three-dimensional-appearing  button.  Most  often  this  action 
is  traversing  a  link  to  another  node  but  it  may  perform 
other  functions,  such  as  terminating  the  hypermedia  program. 
Figure  6  shows  a  screen  from  POPS  which  contains  both 
adorning  and  interactive  graphics. 

Responsive  graphics  can  be  incorporated  into  any  static 
graphic  display  by  designating  a  sensitive  area  or  "hot 
spot."  So,  for  example,  the  labels  for  vehicle  types  in 
Figure  4  or  the  terrain  feature  labels  in  Figure  3  could  be 
linked  to  textual  annotations. 


Dynamic  Graphics 

Two  types  of  dynamic  graphics,  based  on  the  instrumented 
data  of  the  training  exercise  data  set,  have  been 
incorporated  into  POPS.  One  is  a  software  package  adapted 
from  TBAT/MPART  (Nichols  &  Shillcock,  1990)  and  the  other  is 
an  animation  effect  created  by  displaying  an  automatically 
changing  sequence  of  screen  displays.  Both  are  used  in  POPS 
to  support  visualization  of  the  vehicle  positions  as  a 
function  of  time. 

Figure  7  shows  a  screen  from  the  TBAT/MPART  software 
intended  to  provide  a  trace  of  the  battle.  The  software 
replays  all  of  the  available  player  position  data  in 
specially  prepared  files  and  through  color  encoding 
preserves  a  history  of  where  players  of  both  sides  were 
located.  The  display  is  shown  on  a  three-dimensional 
representation  of  the  terrain  viewed  from  the  perspective  of 
a  static  point  above.  The  learner  can  both  pause  and  repeat 
the  replay  without  restriction. 

The  other  dynamic  graphic  system  uses  a  sequence  of  graphic 
files,  generated  by  existing  external  software  that  scales 
and  replays  NTC  instrumented  player  location  data.  These 
files  are  then  displayed  to  the  learner  as  a  sequence  by  the 
authoring  system  to  show  player  movement.  Annotations  are 
added  by  specially  developed  software  described  on  page  9. 
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Figure  6.  Screen  illustrating  both  adorning  and  interactive  graphics. 


Hit  a  key  to  pause. 
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Figure  7.  Screen  from  dynamic  graphic  battle  trace  software. 


The  entire  sequence  of  graphic  screens  is  placed  under  the 
control  of  the  learner  for  self-paced  advancing  or  backing 
up  of  the  displays.  Figures  8  through  11  show  a  sequence 
from  such  a  graphical  display  in  POPS.  It  shows  players  of 
the  training  force  occupying  key  terrain  in  the  vicinity  of 
Hill  780  and  engaging  and  knocking  out  of  play  opposing 
force  players  as  they  approach  from  the  East . 

On  286 -based  computers  with  65  ms  mean  access  time 
Winchester  drives  (relatively  slow)  frames  are  changed  in 
the  automatic  mode  at  a  rate  of  a  little  faster  than  one  per 
second.  A  faster  computer,  especially  with  a  faster  drive, 
would  improve  this  performance. 

Evaluation  and  Feedback  Component 

The  software  developed  for  the  evaluation  and  feedback 
component  of  POPS  met  all  of  the  specifications  described  in 
the  software  section  above.  Responses  of  the  learner  are 
restricted  in  POPS  to  fixed  options.  The  decision 
simplified  the  development  of  the  information  presented  the 
learner  as  feedback.  Figure  12  shows  a  screen  from  the 
evaluation  and  feedback  component  of  POPS.  It  shows  a 
simple  question,  the  learner's  response,  and  feedback  from 
POPS  for  the  particular  response. 

Practically  speaking  there  does  not  seem  to  be  a  limit  on 
how  much  computer  intelligence  can  be  added  to  this 
component,  although  the  latest  opinion  of  the  relevant 
technologies,  including  so-called  artificial  intelligence, 
is  that  they  have  been  over-sold  (Wilkes,  1992)  . 

DISCUSSION 

This  project  led  to  several  conclusions  about  using 
hypermedia  technology  for  training  military  tactics  by 
employing  extant  NTC  data.  It  is  feasible  to  combine  the 
data  and  technology  for  training  purposes;  the  concept  of 
the  project  is  sound.  However,  it  seems  clear  that  while 
most  of  the  data  is  usable,  some  filling  of  gaps  in  the 
training  exercise  story  through  the  generation  of  simulated 
data  will  be  necessary  to  provide  the  learner  with  a 
complete  picture  of  the  training  event.  The  need  to 
minimize  the  risk  of  erroneous  inferences  and  inter¬ 
pretations  in  producing  such  simulated  data  underscores  the 
necessity  of  including  staff  with  expertise  in  NTC  and  Army 
training  on  the  project  development  team.  Further,  the  full 
development  of  the  training  system  will  be  difficult, 
requiring  a  range  of  talents  and  sufficient  resources  and 
appropriate  institutional  support.  In  the  paragraphs  which 
follow,  attention  will  first  be  given  to  some  of  the  issues 
of  the  technology  which  remain  unresolved  and  then  to 
considerations  of  the  resources  which  are  necessary  for  full 
development  of  the  hypermedia -based  training  system. 
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Figure  8.  First  screen  of  dynamic  graphic  sequence. 
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Figure  10.  Third  screen  of  dynamic  graphic  sequence. 
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Figure  12.  Sample  screen  from  Evaluation  and  Feedback  component. 


During  the  several  decades  during  which  the  presence  of 
computers  has  increased  in  the  workplace,  a  great  deal  of 
information  has  accumulated  about  how  humans  and  computers 
interact  (Laurel,  1990).  Increasingly  computers  have  been 
used  successfully  for  training  purposes.  For  example,  in 
1983,  IBM  was  faced  with  rapidly  escalating  training  costs 
and  began  a  major  restructuring  of  its  corporate  education 
programs  using  computer-based  training,  interactive 
videodisc,  and  video  conferencing  on  a  wide  scale.  The 
training  delivery  approach  developed  has  become  a  model  for 
cost  effective  employee  education  throughout  the  world 
(Bowsher,  1989) .  Common  sense  and  evidence  suggest  that 
nothing  about  the  computer  is  inherently  superior  to  other 
instructional  media  (Hannafin  &  Rieber,  1989) 

Specifically  concerning  the  hypertextual  organization  of 
information,  it  has  been  demonstrated  that  novices  find  on¬ 
line  hypertext  easy  and  effective  to  use  for  finding  facts 
(Marchionini  &  Shneiderman,  1988)  and  that  a  hypermedia 
environment  can  facilitate  advanced  learning  by  allowing  the 
users  to  formulate  their  own  knowledge  representations  and 
to  perfect  them  while  exploring  the  content  of  the 
hypermedia  structure  (Lacatis,  Letourneau  &  Banvard,  1989) . 
But  merely  computerizing  something  does  not  necessarily  make 
it  better.  For  example,  drastically  altering  the  structure 
of  text  in  a  journal  article  for  the  hypertext  version  did 
not  aid  the  reader  (Verreck  &  Lkoundi,  1990).  Further, 
numerous  interactive  systems  have  required  specialized 
training  (MacKnight,  1991)  and  often  novice  users,  and 
occasionally  experienced  users,  report  a  disorientation  as  a 
consequence  of  non-sequential  movement  through  large 
hyperdocuments  (Bevilacqua,  1989) .  This  phenomenon  has  been 
termed  the  'lost  in  hyperspace'  problem. 

Then  there  are  graphics.  Are  they  effective  in  contributing 
to  learning?  For  example,  graphic  animation  has  been  found 
to  aid  learners  in  visualizing  complex  concepts,  but  it  has 
been  found  to  have  little  impact  on  learning  without 
integration  into  a  larger  instructional  context  (Rieber,  L. 
1990)  . 

The  work  here  has  been  intended  as  a  feasibility 
demonstration  of  what  is  possible  and  a  determination  of 
what  is  required  for  full  system  development  rather  than  an 
evaluation  of  effectiveness.  Business  and  industry  are 
recognizing  the  need  to  change  the  methods  of  evaluation  for 
interactive  systems  and  multimedia  because  traditional 
methods  based  on  verbal  methods  are  simply  not  appropriate 
for  evaluation  of  visual  processes  (Marlino,  1990)  The 
development  of  POPS  demonstrates  that  interactive  hypermedia 
technology  exists  today  to  produce  quality  instructional 
materials  of  moderate  complexity  for  training  fundamental 
tactics. 
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To  comprehensively  evaluate  this  technology  in  terms  of  its 
effectiveness  in  comparison  to  traditional  tactical  training 
methods  has  been  beyond  the  scope  of  the  current  effort. 
However,  this  is  research  which  needs  to  be  done  prior  to  a 
resource- intensive  full  development  effort  is  undertaken. 
Such  an  evaluation  if  properly  planned  and  conducted  may 
also  serve  as  the  basis  for  a  thorough  cost-benefit  analysis 
of  various  methods  for  training  fundamental  tactics. 

What  is  needed  for  a  full  development  of  the  concept 
demonstrated  by  POPS?  As  with  most  projects  the  key  to  a 
successful  development  of  the  interactive  hypermedia 
tactical  trainer  will  depend  upon  the  planning  phase.  Solid 
sponsorship  and  elucidation  of  project  goals  and  objectives 
will  be  critical.  Quality  is  expensive  but  is  often  a 
savings  in  the  long  run. 

The  complexity  of  computer-based  training  development  should 
not  be  underestimated.  Fourteen  distinct  roles  involved  in 
the  development  process  of  the  typical  computer-based 
trainer  have  been  identified  (Gery,  1987) .  These  include 
Project  Manager,  Program  Sponsor,  Instructional  Designer, 
Subject  Matter  Expert,  Writer,  Editor,  Data  Entry 
Specialist,  Authoring  System  Host  Computer  Programmer,  Media 
Expert,  Graphics  Designer,  Authoring  System  Specialist, 
Learner  Evaluator,  Production  Administrator,  and  Trainer 
Administrator.  Assembly  of  the  development  team  and 
resourcing  it  adequately  are  critical  for  project  success. 

Computer-based  instruction  development  ratios  reported  in 
the  literature  range  from  1  to  4000  hours  of  development 
time  for  every  hour  of  instruction  with  modal  dominance  in 
the  200  to  350  range.  Reasons  for  high  costs  vary,  most  are 
situationally  dependent  (Orlansky  &  String,  1979;  Jay, 

1989).  Gery  (1987)  presents  the  basis  for  a  more  refined 
model  of  cost  estimation  for  a  computer-based  trainer 
development.  Fourteen  courseware  variables,  five 
technological  factors,  twelve  human  variables  and  five 
additional  factors  impact  the  cost  and  development  time. 
Collectively  these  factors  yield  a  range  of  from  85  to  300+ 
development  hours  per  computer-based  training  hour. 

Judging  the  complexity  of  POPS  on  the  relevant  dimensions 
produces  an  estimate  near  the  low  end  of  the  range. 
Substantial  enhancement  of  the  evaluation  and  feedback 
component  as  discussed  above  could  be  expected  to  move  the 
cost  toward  the  center  of  the  range.  Adding  video  and  audio 
components  would  likely  have  costs  toward  the  higher  end  of 
the  range.  Further,  a  shorter  lesson,  4-8  hours  of 
instruction,  could  be  expected  to  have  higher  per  hour 
development  costs  than  a  longer  lesson  of  60-80  hours  of 
instruction . 
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CONCLUSION 


Indiscriminate  application  of  technology  is  not  the  answer 
to  the  challenges  of  Army  training.  However,  judicious  use 
of  information  technology  holds  exhilarating  potential. 

This  has  been  expressed  succinctly,  "because  information 
technology  moves  so  very  much  more  quickly  than  other  kinds  of 
technology — halving  in  price  and  doubling  in  power  every  two 
years  on  the  average"  (Feigenbaum,  McCorduck,  &  Nii,  1988) . 

Finally,  interactive  hypermedia,  and  hypertext  more  generally, 
may  be  viewed  as  one  stop  along  the  technological  roadway  toward 
training  worlds  based  on  virtual  reality  (Krueger,  1991) .  Such 
training  worlds  in  which  the  learner  is  immersed  in  a  simulated 
sensory  environment  to  maximize  learning  and  transfer  to  the  real 
thing  have  been  identified  as  particularly  appropriate  (1)  in 
hostile  environments,  (2)  where  there  is  a  high  benefit  from 
rehearsal  and  practice,  and  (3)  where  there  is  high  potential  for 
damage  to  equipment  or  danger  to  individuals  (Middleton,  1991) . 
These  characteristics  accurately  describe  combat  on  the  modern 
battlefield. 
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